1. Field of the Invention
The present invention relates to an automatic monitoring system, and more particularly to a method of and an apparatus for detecting a moving object utilizing a technology of extracting from a motion image a moving area where a movement (of the object) has occurred.
2. Description of Related Art
Nowadays, security systems, which each automatically detect entering/leaving of vehicles into and from a facility, such as a parking lot, using a video camera, have been the individual facilities. In this conventional method, motion image data of a moving object is extracted from the individual scenes obtained from data of a motion image taken by a video camera. As this extraction method, a background subtraction method is currently known.
In the background subtraction method, a moving object is detected by comparing a motion image (input image) with a background image of the background. FIG. 24 of the accompanying drawings illustrates the background subtraction method using an input image 203a, a background image 203b, and a background difference image 203c. The background image 203b is subtracted from the input image 203a, which is a composite image composed of a moving object 204 and the background image 203b, to obtain the background difference image 203c. 
Specifically, a difference, between brightness values of the input image 203a and previously recorded brightness values of the background image 203b located at the same position as the input image 203a, is calculated. If the calculated difference is equal to or higher than a particular threshold, it is judged that a change in brightness value has occurred due to appearance of a moving object 204. Thus the moving object 204 is detected.
The term “brightness value” is a value representing a degree of fullness of light and shining per pixel in an image. The term “pixel” is a minute area on the display screen which area corresponds to a resolution of the image. Additionally, in the following description, the brightness value will also be called simply “brightness”; the object which moves, “moving object”; the motion image, “image”.
The background subtraction method has a problem in that mis-detection would tend to occur due to an environmental change, such as a change in weather, turning on and off the light or changeover of illumination. FIGS. 25(a) through 25(c) are graphs illustrating the manner in which a mis-detection occurs due to an environmental change according to the background subtraction method. In the individual graph, the horizontal axis represents positions on the screen, and the vertical axis represents brightness values in these positions. In FIG. 25(a), two brightness distribution curves 205a, 205b are shown. The brightness distribution curve 205a is a string of brightness values of the original image (screen), while the brightness distribution curve 205b is a modified form which the brightness distribution curve 205a has been modified because light has been turned on.
As the brightness distribution curve 205a of FIG. 25(b) is subtracted from the brightness distribution curve 205b of FIG. 25(a), it produces a brightness distribution curve 205c corresponding to the difference in brightness value between the input image and the background image so that an environmental change would be mis-detected as a moving object.
Consequently, in order to avoid mis-detection due to an environmental change, a brightness distribution shape comparison method is used. In this comparison method, appearance/absence of an object is detected by dividing the image into a plurality of blocks, each composed of a number of pixels, and measuring a difference in shape of brightness distribution in each block. In the following description, the individual block is called “unit block” or “sub-area”. This comparison method will now be described more specifically with reference to FIGS. 26(a), 26(b), 27(a) and 27(b).
FIGS. 26(a) and 26(b) illustrate a change of brightness occurs due to an environmental change. The input image 203a of FIG. 26(a) is a monitoring region composed of two areas A, B. The area A represents an area shaded from sunlight, and the area B represents an area bright with sunlight.
The change of brightness due to environmental change is not uniform in the monitoring region (corresponding to the input image 203a). For example, in each of the areas A, B, the change of brightness is not uniform. In a limited local area, this change of brightness can be regarded as being uniform. For example, the brightness change in a limited local area indicated by a white circle in the area B in FIG. 26(a) is as shown in FIG. 26(b). Two brightness distribution curves 205d, 205e of FIG. 26(b) represent respective brightness changes in the circled local area of the area B. The brightness distribution curve 205d represents a brightness distribution when the circled local area is bright with sunlight, and the brightness distribution curve 205e represents a brightness distribution when the circled local area is shaded from sunlight.
On the other hand, FIGS. 27(a) and 27(b) illustrate a brightness change around a moving object. The input image 203a of FIG. 27(a) has a moving object 204. The brightness distribution in a ridge 210 of the moving object 204 is different in shape from the brightness distribution of the background image as shown in FIG. 27(b).
Therefore, by dividing the monitoring region into small unit blocks and obtaining a difference in shape of distribution of brightness values in the unit blocks between the input image and the background image, it is possible to detect appearance/absence of an object in terms of the individual unit blocks, irrespective of environmental changes.
This method is exemplified by “MOVING OBJECT DETECTION BY TIME-CORRECTION-BASED BACKGROUND JUDGEMENT METHOD” by Nagaya, et al., The Institute of Electronics, Information and Communication Engineers Theses Review (D-II), Vol.J79-D-II, No.4, pp. 568-576, 1996 (hereinafter called Publication 1), and “A ROBUST BACKGROUND SUBTRACTION METHOD FOR NON-STATIONARY SCENES” by Habe, et al., Meeting on Image Recognition and Understanding, MIRU '98 (hereinafter called Publication 2).
In a technology disclosed in Publication 1, brightness values in blocks are regarded as vector factors, and a normalized inter-vector distance of the input image is compared with that of the background image.
And in a technology disclosed in Publication 2, a variance in inter-block local distance of the input image is compared with that of the background image. In the following description, the term “enter” means that an object have entered into a monitoring region, and the term “leave” means that an object leaves from a monitoring region.
Japanese Patent Laid-Open Publication No. HEI 9-81753 (hereinafter called Publication 3) discloses a moving object extracting apparatus which can correctly identify a moving object even when a split of a moving area occurs in the same moving object because there is no adequate difference in brightness and tint between the input image and the background image. In Publication 3, when an object has concealed by another object, the concealing is detected, and then immediately after settlement of the concealing, the settlement is recognized to specify a settlement time, whereupon a shaped part of the intended object is trimmed from the detected moving area to extract a correct partial image of the intended object.
Japanese Patent Laid-Open Publication No. HEI 5-174261 (hereinafter called Publication 4) discloses another moving object detecting apparatus which prevents mis-detection due to occurrence of shading by a moving object detector and due to change in illumination, based on a difference image.
Japanese Patent Laid-Open Publication No. HEI 8-249471 (hereinafter called Publication 5) discloses still another motion image processing apparatus which detects, evaluates and judges a characteristic amount of a moving object to realize a follow-up survey including a stop-status of the moving object despite using an inter-frame difference process. And if parameters determined by given environmental conditions are inadequate, they are soon substituted by optimal parameters to realize precise detection.
Japanese Patent Laid-Open Publication No. HEI 8-167022 (hereinafter called Publication 6) discloses an image monitoring apparatus which enable sure follow-up survey and detection of a moving object at an improved processing rate.
Japanese Patent Laid-Open Publication No. HEI 8-106534 (hereinafter called Publication 7) discloses another object moving detecting apparatus which detects a moving object in a monitoring area with accuracy and recognizes the detected moving object.
However, in the object detecting methods described in Publications 1 and 2, detection of an object is made chiefly by detecting a ridge of the object. Therefore, when an object flat in shape of brightness distribution enters an area whose shape of brightness distribution of background is flat, it is impossible to detect a movement of the object from inside of the area where the moving object appears. This will be described more specifically using FIGS. 28(a), 28(b), 29(a), 29(b), 30(a) and 30(b).
FIG. 28(a) schematically shows an input image in which no object appears; the shape of a brightness distribution curve 211 is flat. And FIG. 28(b) schematically shows an input image in which an object appears; the object is a vehicle, for example, and the input image has areas 200a, 200b, 200d. In the areas 200a, 200d, since change in brightness distribution shape is large as compared to the brightness distribution curve 211 of FIG. 28(a), a movement of the object can be extracted.
In the meantime, in the area 200b which occupies an inner portion of the object-appearing area partly since the shape of the area 200b is analogous to the flat shape of the brightness distribution curve 211 of FIG. 28(a), a movement of the object cannot be detected. Since the real size of the object is not always proportional to the number of moving blocks to be detected, the following problems would occur.
FIG. 29(a) schematically shows an input image in which a single large-sized object, such as a vehicle, has entered. In FIG. 29(a), a brightness distribution curve of the object is flat in its central portion.
FIG. 29(b) is a top plan view showing a monitoring area in which a single large-sized object has entered. The monitoring area 23 is an area extracted from an input motion image. Small rectangles (quadrilateral) designated by 21 are areas, called moving blocks, whose brightness distribution changes have been detected by comparison in terms of unit blocks.
FIG. 30(a) schematically shows an input image in which two small-sized objects, such as two persons, have entered. And FIG. 30(b) is a top plan view showing a monitoring area 23 in which two small-sized objects have entered. In FIG. 30(b), object ridges are many for the size of the moving objects, which is smaller than the example of FIG. 29(b). Since the nearly same number of moving blocks 21 as in the example of FIG. 29(b) would occur, discrimination cannot be made between these two examples.
FIG. 31 is an arrangement diagram of unit blocks and a ridge of an object. In FIG. 31, the ridge 201 of the object extends along an end of rectangular area of the individual unit block 20. Therefore, the change of brightness distribution in the unit block 20 would be so small to detect the object.
FIG. 32 schematically shows an input image when a camera (not shown) has encountered swing or vibration; for example, the camera is fixedly mounted in the open air. Even though the image 202a of FIG. 32 was taken at a normal position, the image of the object would be deviated as the camera angle varied, resulting in an image 202b and hence mis-detection.
Publications 3 through 7 are totally silent about either a technology of detecting two kinds of objects as discriminated or a technology of preventing mis-detection due to swing or vibration of the camera.